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Abstract — 

The Xapagy cognitive arcliitecture had been designed to 
perform narrative reasoning: to model and mimic the activities 
performed by humans when witnessing, reading, recalling, 
narrating and talking about stories. Xapagy communicates 
with the outside world using Xapi, a simplified, "pidgin" 
language which is strongly tied to the internal representation 
model (instances, scenes and verb instances) and reasoning 
techniques (shadows and headless shadows). While not fully a 
semantic equivalent of natural language, Xapi can represent a 
wide range of complex stories. We illustrate the representation 
technique used in Xapi through examples taken from folk 
physics, folk psychology as well as some more unusual literary 
examples. We argue that while the Xapi model represents a 
conceptual shift from the English representation, the mapping 
is logical and consistent, and a trained knowledge engineer can 
translate between English and Xapi at near-native speed. 



I. Introduction 

Xapagy is a cognitive architecture focusing on narrative reasoning. 
The objective of the system is to mimic the human behavior with 
regards to reasoning about stories. The reasoning in Xapagy is based 
on autobiographical memory - newly encountered events or story 
snippets are shadowed by elements of the memory, while headless 
shadows project into the future, providing prediction or in the past, 
providing detection of elements which might be missing in the current 
narrative. 

The interaction of the Xapagy agent with the outside world happens 
through the Xapi pidgin language, a language with a simplified 
syntax, which approximates closely the internal representational 
structures of Xapagy. In the following we will provide a brief primer 
on Xapi and the internal structures it represents. 

A Xapi story is a sequence of sentences. Sentences can be in 
the simple subject-verb-object, subject-verb or subject-verb-adjective 
form. A single more complex sentence exists, in the form of subject- 
communication verb-scene-quote, where the quote is an arbitrary 
sentence which is evaluated in a different scene. Subjects and objects 
are represented as instances which can acquire various attributes in 
form of concepts. In general, each sentence corresponds to an internal 
Xapagy structure called a verb instance. Verb instances are stored in 
the autobiographical memory in a raw form. They can, however, be 
linked with succession, summarization, context and coincidence links. 

As we shall see, one of the unexpected features of Xapagy 
instances is that they do not exactly correspond to entities in human 
speech. An entity in colloquial speech is often represented with more 
than one instance in Xapagy. These instances are often connected 
with identity relations but participate independently in verb instances, 
shadows and headless shadows. 

Xapagy uses autobiography to perform reasoning, it can not reason 
on logical basis or first principles. In conclusion, the quality of the 
reasoning of an agent depends on the quality and representativeness 
of its autobiography, which includes direct and indirect experiences, 
stories which had been witnessed, heard or read. As we have a 



relatively large set of stories available in narrative form, translating 
stories from English to Xapi is an important step in the construction 
of a synthetic autobiography. 

Xapi, as a "pidgin" language, uses an English vocabulary. Stories 
described as simply a series of consecutive actions are trivial to 
translate. Let us assume the following snippet placed in the world 
of the Iliad: 

Hector is a Trojan warrior Achilles is a Greek warrior 
Achilles has a shield. Achilles 's shield is big. Achilles hits 
Hector Hector strikes Achilles. 

This can be translated into Xapi as follows: 

1 A "Hector" / exists. 

2 "Hector" / is-a / trojan warrior. 

3 An "Achilles" / exists. 

4 "Achilles" / is-a / greek warrior. 

5 "Achilles" / owns / a shield. 

6 The shield — of — "Achilles" / is-a / big. 

7 "Achilles" / hits / "Hector", 
s "Hector" / hits / "Achilles". 

There are a number of situations where English to Xapi translation 
is not possible. One such case is self-referential English sentences 
such as "This sentence has five words". Similarly, sentences where 
the reference is made on the expressive form unlikely to be fully 
semantically translatable. For instance, the statement The song "Love 
... thy will he done" by Prince is a religious parable applied to love. 
is self-evident in English, due to the archaic English form chosen in 
the title. Naturally, such subtleties are not present in Xapi. In general, 
everything which is difficult to translate, let us say from English to 
French, would be near impossible to translate from English to Xapi. 

The objective of this report is to help human operators translate 
English into Xapi by offering a cookbook of translation solutions for 
specific situations. As a note, for reasons of conciseness, the Xapi 
snippets will sometimes be abbreviated to the most interesting parts. 
The full Xapi stories for the specific translations are downloadable 
form the Xapagy webpage. 

The remainder of this report is organized as follows. Section ?? 
discusses trivial cases of representation and identify the nature of 
challenges which are encountered when translating more complex 
stories. 

Complex semantic structures are represented in English using 
complex syntax. Xapi sentences have a very simple syntax, thus 
the semantic of the story must be expressed through other tools, in 
particular by scenes and scene relations, identity relations between 
instances and the notion of change and coincidence links. This is 
covered in Section HTl 

Section lin] discusses specific solutions for representational chal- 
lenges covering the area of folk physics, while Section [TV] deals with 
examples drawn from folk psychology. 

As a note, the Xapagy system is actively developed and not all the 
representational challenges are solved. In this report we will describe 
future work is separate notes using italic text. 



II. Representation tools in Xapi 

We have seen that translation is trivially possible when we have 
simple sequences of consecutive actions and we have seen that there 
are cases where fully semantic translation is unlikely to be possible. 
In between these two cases, there is the wide range of cases where 
fully semantic translation is possible, but it requires a translation from 
the grammatical concepts to the way Xapagy thinks about things. 

The reader will find the Xapagy internal structures somewhat 
counter-intuitive, and indeed, the shift from the text to the Xapagy 
internal representation (and Xapi) requires a certain non-trivial shift 
of perspective. This is similar to the way in which a linguist considers 
a sentence in terms of its syntax tree - a representation which is 
counter-intuitive for non-specialists. 

In the following we will discuss four tools of the Xapagy / Xapi 
representational model which allow us to represent significantly more 
stories than the trivial ones we discussed before. These are the (a) 
scenes and scene relations (b) identity relations (c) change and (d) 
coincidence. 

A. Scenes and scene relations 

Human languages do not limit themselves to the enumeration of 
events witnessed in the current moment. They deal with the past, 
future, hypothetical situations, plans and so on. These are represented 
in language in form of categories such as tense, modality, voice and 
so on. 

Such situations are expressed in Xapagy by placing the actions and 
the instances which participate in them in different scenes. Instances 
in different scenes are often connected through identity relations. 
The grammatical category is then represented using the relationship 
between the scenes. 

The most unusual part of this representation is the fact that we 
have instances replicated in the different scenes. 

If we consider a situation when Achilles remembers his fight with 
Hector, the common sense interpretation tells us that we are talking 
about a single Greek warrior. The Xapagy representation, however, 
involves two instances of Hector in different scenes, with possibly 
different attributes and, naturally, with different shadows. 

In conclusion, the task of the translator when encountering a 
specific grammatical category in English is: 

- identify what instance in what scene it refers to 

- if necessary, create a new scene, new instance and set up the 
appropriate identity relations. 

B. Identity relations - further uses 

We have seen the use of identity relations to express grammatical 
categories such as tense and modality. This can be (in theory) 
automatically generated using syntactic parsing. For instance, if we 
determine that some parts of a text are in past tense, we put it in a 
past scene. 

Identity relations between instances are used in some less obvious 
cases as well, where the scenes represent situations which in the 
English text are encoded in the semantics, not in the syntax. 

Let us consider the famous dialog between Little Red Riding Hood 
and the wolf dressed up as Grandma. There are two planes of inter- 
pretation: one involves a grandma-granddaughter conversation, while 
the other one is a predator-prey interaction. These have naturally 
different interpretations, thus they need to be shadowed differently. 
The Xapagy representation, as expected, involves two scenes: one 
involving LRRH with Grandma, while the other one LRRH and the 
wolf. 

From the point of view of the human translator, she must identify 
situations when a certain story snippet involves multiple views and 



represent them with the appropriate number of scenes and identity 
relations. Such situations often involve cases where the different 
instances have different attributes (sometimes, as in the case of the 
wolf disguised as Grandma, radically so). 

It is also possible that one of the instances is further connected 
with an identity relationship to somewhere else, but not the other. 
In our case. Grandma imagined by LRRH is connected to the real 
Grandma from other scenes, but not the wolf. 

C. Change 

The Xapagy framework posits that the acquisition of new attributes, 
for an instance is an act of discovery not change. If we say: 

1 "Hector" / is-a / trojan. 
it does not mean that he had just became a Trojan, but that he was a 
Trojan all the time and we have just discovered it. In conclusion, the 
attributes which can be added with is -a must be cummulative, they 
must be compatible with the previous attributes. Naturally, however, 
the new attributes added through is -a will affect the shadows and 
headless shadows in the future. 

What colloquial speech perceives as a change in the attributes of an 
entity, in Xapagy is represented by a verb which creates a separate 
new instance, which is identity-related to the old instance, which 
carries over some of the attributes of the old instance but also has 
new, possibly incompatible attributes: 

1 "Hector" / changes / dead. 
The intuition behind this approach is that the two instances will 
have radically different shadows - the live Hector will be shadowed 
by other warriors, other Trojans and so on, while the dead Hector 
will be shadowed by other corpses. Based on this, the Xapagy system 
can make appropriate predictions of what Hector will do (which are 
obviously different for Hector the warrior and Hector the corpse). 

D. Coincidence links 

Coincidence links connect multiple Vis together. The intuition 
behind coincidence links is that these Vis describe different aspects of 
a certain real world event. Sometimes coincidence is used to describe 
different participants in an event (when this cannot be done using 
only a subject and object). In other cases, it is used to represent 
the relation between a physical event and its significance in another 
plane of thinking (which would be likely represented in another 
scene). Coincidence links are created by adding the thus verb to 
the sentences which must be connected to a previous sentence, using 
labels to resolve ambiguous cases. 

III. Folk physics 

This section contains examples of representation situations which 
we can label as the general term of folk physics. This includes things 
such as instrumentality, ownership, part relations and number sense. 

A. Ownership exchange 

Ownership in Xapagy is represented with a relation between the 
owner and the owned entity. The same concept applies to groups 
of relations such as physical containment, physical attachment, legal 
ownership and so on. The relationship is considered active as long 
as the relation VI is in focus. A relation VI can be expired from the 
focus, and a similar relation added later. 

Let us now discuss how we can represent the exchange of owner- 
ship, for instance in the following snippet: 
Johnny gives an apple to Mary. 

This implies that there was an apple, which is in the ownership of 
Johnny. This is not an action, only a situation: 



Scene: [] #current 



"Johnny" / disowns / 
apple. 



"Mary" / thus 

receives / the 

apple 



Scene #current / 

create-instance / 

apple. 



Fig. 1. Focus Vis for representing ownersliip excliange. 
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"Achilles" / create 

-relation 

LegallyOwns / 

sword. 








"Hector" /hits/ 
"Achilles" 




"Achilles" / cuts / 
"Hector" 




"Achilles" / thus 
uses / the sword 
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Fig. 2. Instrumentality represented as a coincidence group and the "uses" 
verb. 
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] #Troy 








"Hector" / hits / 
"Achilles" 




"Achilles" / 
achieves 


"Hector" / thus 
changes / dead 
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1 An apple / exists. 

2 "Johnny" / has / the apple. 

3 "Johnny" / gives / an apple. 

4 "Mary" / thus receives / the apple. 

The other challenge is that the English sentence has a subject 
(Johnny), a direct object (apple) and an indirect object (Mary). This 
can not be represented with a single VI, which at most allows a 
subject and an object. 

The representation model splits the English statement into two Vis, 
one for Johnny giving the apple and the other one for Mary receiving 
it, but connects them with a coincidence link, which shows that these 
two Vis describe different aspects of a single event. Figure [T| shows 
the focus Vis for this exchange. 

One of the consequences of this representation model is that we can 
represent situations where we know only one half of the statement. 
For instance, we can say: 
Mary is given an apple. 

which is translated: 

1 An apple / exists. 

2 "Mary" / receives / the apple. 

The second VI will be shadowed by sentences of the given form, 
some of them having coincidence links to the VI pair which specifies 
that there is somebody who gave the apple. In the course of the 
normal operation of the Xapagy agent, this will appear as headless 
shadow suggesting a missing or predicted action - depending on other 
settings, the agent might or might not act on these suggestions. 

B. Instrumentality 

Instrumentality is the situation where an agent uses an external 
entity as an instrument to accomplish an action. This is a very 
important component of many conscious actions involving humans. 
Linguistically, instrumentality is shown in English using a preposi- 
tional object: 

Achilles cuts Hector with the sword. 

The challenge in translating this to Xapi is that this is clearly a 
single action, which, however, involves three entities which can not 
be fit into a single VI: 

In Xapagy instrumentality is represented by an S-V-0 VI using 
the verb vm_Uses with the subject being the agent which uses the 
instrument and the object being the instrument used. This verb is often 
part of a coincidence group which describes the complex action: 



Fig. 3. 
verb. 



Achievement represented as a coincidence group and the "achieves" 



1 "Achilles" / cuts / "Hector". 

2 "Achilles" / thus uses / the sword. 

The resulting focus Vis are shown in Figure [2] 

C Agency and achievement 

A frequently used language construct designates an entity as being 
the agency who brought about a certain state of the world, without 
explicitly describing the actions it used to do so. 

Narratives involving humans use this construct frequently. At the 
linguistical level, such constructs usually use the same very general 
verb (gets, puts, makes and so on) although in some cases, we have 
more specific verbs which depend on the achieved state (e.g. kills, 
destroys, repairs and so on). 
Achilles kills Hector. 

In Xapi, this representation is achieved by a coincidence group in 
which the subject agent is the subject of a verb achieves. 

1 "Achilles" / achieves. 

2 "Hector" / thus changes / dead. 

D. Parts and lack of them 

An important aspect of folk physics is the reasoning about parts 
/ whole relationships. These are represented by relationships in 
Xapagy, the basic part relationship being provided by the verb 

contains. 

We can say that Billy has a head by: 

1 A "Billy" / is-a / man. 

2 "Billy" / contains / a head. 

The fact that most humans have a head is part of the autobio- 
graphical information of the agent. This needs to be put in contrast 
to default logic or frame systems, where there is a logical description 
of the structure. 

In most stories, we do not need to enumerate the body parts 
of the agents. With a different autobiography, the Xapagy agent 
will maintain them as a headless shadow, thus when we refer to 
the head — of — Billy, the instance will be generated auto- 
matically. 



Future work: this create on demand by reference is present in 
Xapagy but it is largely untested and used very rarely. It should get 
more use in larger autobiographies. 

The next question is liow do we represent tlie fact tliat a certain 
part is missing, despite tiie fact thiat it would be predicted by the 
autobiography? The fact that it had not been enumerated in the current 
scene is not sufficient to indicate its absence. 

Let us consider that we want to talk about the headless horseman. 
We need to represent the absence of the head explicitly. Simply not 
mentioning the existence of the head is not enough, as we have seen, 
for most humans the stories do not explicitly mention the existence 
of the head, yet we usually assume its presence. 

This is accomplished by explicitly creating an instance for the 
head which has the inexi stent attribute. This instance will be put 
in the appropriate part relation with the human. This would allow 
it to be explicitly shadowed by other headless humans from the 
autobiography: 

1 A "HeadlessHorseman" / is-a / man. 

2 "HeadlessHorseman" / contains / an inexistent head 

E. Number sense: none, one, pair and many 

The Xapagy agent has a simple number sense which allows the 
representation of four classes of numbers: none, one, pair and many. 
In the following we will describe the semantics and representation 
of these concepts. 

None: represents the absence of instances of a certain kind of objects 
in a certain context. For instance, we might say that there are no 
pencils in a box. This is represented through an inexistent instance 
as seen in the previous example. 

1 A box / exists . 

2 The box / contains / an inexistent pencil. 

One: represents the existence of exactly one instance of a certain 
kind in a certain situation. For instance, we might say that there is 
one pencil in the box. This is represented as an instance directly 
mapping to the object. 

1 A box / exists . 

2 The box / contains / a pencil. 

Pair: represents the existence of exactly two instances in a certain 
situation. This is represented through a group instance, which also 
has the pair attribute, with the two instances being members of the 
group. 

1 A box / exists . 

2 The box / contains / a pair pencil . 

3 A pencil red / becomes-right-of / pair. 

4 A pencil blue / becomes-left-of / pair. 

Many: represents a large number of instances in a certain situation. 
This is represented through the a group instance which also has the 
many attribute. The semantics of the many groups is that they are 
not enumerated - it is generally assumed that not all the members 
are individually 

1 A box / exists . 

2 The box / contains / a many pencil . 

3 The many pencil / contains / a red pencil. 

F. Noun phrase modifiers and identity over time 

Most noun phrases are centered in Xapagy around a single instance. 
However, there are cases where the Xapi representation is counter- 
intuitive, as the specification provided by the modifier phrase takes 
place in a different scene: 

The apple which was picked by John yesterday is red. 
Commonsense interpretation would posit the existence of a single 
apple. In Xapagy, however, we are talking about two scenes: one of 




Fig. 4. A noun modifier phrase refers to a different instance of the apple. 



them the present reality, while the other the scene of yesterday. As 
every instance is member of a single scene, the implication is that we 
have two instances of the apple, connected with an identity relation: 

1 $NewSceneOnly #current, none, apple 

2 $NewSceneCurrent tyesterday, none, 

3 apple-> apple, "John" 

4 Scene #yesterday / is-past-of / scene #current. 

5 Scene #yesterday / is-yesterday-of / 
. 6 scene "current". 

7 John / picks / the apple. 

8 Scene #current / is-current-scene . 

9 The apple / is-a / red. 

The resulting instance structure in the focus is shown in Figure H] 

IV. Folk psychology 

Folk psychology covers the way in which everyday people formal- 
ize the attribution of mental states to themselves and other humans, 
as well as the relationship between the humans' mental states and 
their past and future actions. Folk psychology is an important part of 
many stories, as some of them explicitly use the constructs of folk 
psychology as story elements. 

In this section we describe several examples of translating story 
snippets involving folk psychology from English to Xapi. 

A. Understanding perceptions 

I am watching Jay Leno and Brad Pitt talk on the television. 
Jay Leno says something. Brad Pitt laughs. I realize that 
Jay Leno was joking. 

There are several representational challenges here. First, there are 
two narrative spaces: the talk show, where Jay Leno and Brad Pitt 
are present, and my room, where (as far as this snippet goes) I am 
the only one present. This is naturally represented using two scenes 
in Xapagy, with the perception in scene #Television appearing not 
directly, but through a quote-type construction through the character 
Me in the #Reality scene. 

The other challenge is that the television perception and inference 
must be mixed up: events in the scene #Television include things 
which the character Me sees, and some which he only infers. In 
the case of the joking of Jay Leno, we have a situation where the 
coincidence group had been formed from sentences which had been 
directly witnessed, and sentences which have been inferred. This is 
a very typical construct in Xapi stories. 

Finally, the last challenge from the point of view of creating a 
Xapi text is that the interpretation was created not immediately after 
the witnessed action, but later. This means that we cannot use the 
simple version of the thus coincidence creation word. Instead we 
need to use a labeled version which allows us to refer back to a 
previous statement. Note that this kludge is not necessary if the 
interpretation is created internally through a headless shadow, which 
contains references to objects. 

The resulting Xapi code looks as follows: 

1 $NewSceneCurrent #Reality, none, man "Me" 

2 $NewScene #Television, none, man "JayLeno", 
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Fig. 5. The Vis in the focus after a series of perceptions where the 
interpretation is added to the perception later. 



3 man "BradPitt" 

4 "Me" / sees in #Television // 

5 "JayLeno" / talks-to #A / "BradPitt". 

6 "Me" / sees in #Television // 

7 "BradPitt" / laughs. 

8 "Me" / thinks in #Television // 

9 "JayLeno" / thus #A jokes. 

Parsing this Xapi story snippet creates a VI structure in the focus 
illustrated in Figure [5] Note how the perception and thinking actions 
are in a natural succession of the statement. However, this succession 
is different in the #Television scene where the interpretation snaps 
back to become coincidental with an earlier perception. 

As a note, such delayed interpretation can occur internally in 
Xapagy when a statement is created as a missing action. 

B. Trying, succeeding and failing 

Human language frequently deploys constructs which show that 
an agent attempts to achieve a certain state of the world. Naturally, 
such story snippets must describe the given state of the world. Let 
us consider an example: 

George tries to put a cat in the box. 

The challenge in translating this statement to Xapi is that we need 
to describe a situation where the cat is in the box, which, however, 
is not necessarily true. The Xapagy model requires two scenes and 
two instances of the cat and the box to represent this. The following 
snippet implements this: 

1 $NewSceneCurrent #Reality, none, man "John", 

2 cat, box 

3 $NewScene #Attempt, fictional-future, 

4 man "John" -> man "John", 

5 cat -> cat, box -> box 

6 "John" / tries in #Attempt // 

7 "John" / achieves. 

8 "John" / tries in #Attempt // 

9 The cat / thus is-inside / the box. 

The resulting focus instances can be seen in Figure [6] 

While it goes beyond the scope of this technical report, let us 
quickly investigate what kind of predictions or inferences will Xapagy 
make in this situation? The HLS framework will recognize the 
specific "trying" structure in terms of achievement in a fictional 
future, and it can generate headless shadows for both success and 
failure. 

Success is simply a transposition of the achieves status in the 
fictional future into the current state: 

1 The cat / is-inside / the box. 
while failure is its negation: 
1 The cat / not-is-inside / the box. 

Finally, the code snippet above uses the specific Xapi L2 macro to 
create the attempt scene and set up the identities. Xapagy also has 



a specific meta-verb clone-scene which creates a cloned version 
of the current scene, sets up identity relations and a specific relation 
between the current scene and the newly created scene. 

1 Scene / clone-scene / scene #Attempt . 

This is especially convenient for the inference and recall mecha- 
nism as it allows to implement a frequently used construct concisely. 

C. Planning ahead 

In the previous example we discussed the case of an attempted 
action. A similar situation appears when an agent makes a possibly 
longer plan: 

John was hungry, so he decided to go to the kitchen, get 
some milk from the fridge and drink it. 

Before we move to the Xapi translation, let us first discuss what 
this natural language statement mean to a human observer. We can 
assume that John is not in the kitchen, he is engaged in some activity 
elsewhere. Although the statement describes a number of actions 
taking place, we know that these actions do not actually take place: 
for the time being, John will stay where he is. 

The Xapagy representational model for such situations is to create 
two instances of John. One instance of John, in the scene representing 
the current reality, will stay put and imagine the plan. The other 
instance of John, in the other scene, connected to the current one 
with a future-fictional relation is actually executing the steps 
necessary for the plan. The two instances of John are connected with 
an identity relation. The Xapi code for this sequence will be: 
! $NewSceneOnly #Reality", none, man "John" 

2 "John" / is-a / hungry. 

3 $NewScene #Plan, fictional-future, 

4 "John" -> "John" 

5 "John" / plans in #Plan // 

6 I / goes-to / a kitchen. 

7 $. .// I / open / a fridge. 

8 $. .// A milk / is-inside / the fridge. 

9 $. .// I / pick-up / the milk. 
10 $. .// I / drink / the milk. 

Executing this code will yield the instance and VI structures in 
Figure [7] 

As we are representing a plan, let us discuss what exactly makes 
this structure a plan. The first observation is that the scene #Plan, by 
itself, does not have anything which would identify it as a plan (or 
even fictional). There is no "fictional" marker on the scene. It is only 
in relation to the original scene that the #Plan scene is connected 
with the "future-fictional" relation. One implication of this is that 
we can represent multiple embeddings of scenes, each of them being 
fictional with relation to another. Also note that the scene #Reality 
itself is not the reality: it is only an example in this paper. 

D. Enacting a plan 

Let us now discuss the representation of the enactment of the plan 
created in the previous section. Naturally, enacting a plan is just a 
series of actions. What we are interested in is how the enactment 
in the real scene will relate to the planning scene. The problem of 
how the plan drives the agent's concrete actions is an issue of story 
steering, and us such is beyond the scope of this paper Nevertheless, 
even if we are looking at the agent from the outside, we still need to 
find the correspondences between his plan and its actions. To illustrate 
this internal view, let us discuss the representation of the following 
statement, which is a continuation of the one in the previous section: 
...So John went to the kitchen and opened the fridge. The 
fridge was empty. 

The Xapi representation for this sentence (as following the previous 
ones) is as follows: 



Scene: ["Reality"] 


.. ■ -► 


"John" 
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box 
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Scene: ["Attempt" 



"John" 



Inside 



box 



cat 



Fig. 6. The focus instances after a snippet which describes an unsuccessful attempt to put a cat in the box. In the #Attempt scene, we have a state of the 
world where the cat is in the box, in #Reality, we don't. 



Scene: [] #Reality 



"John" / plans in 

scene #Plan // Scene 

#Plan / create 

-instance / milk. 




"John" / plans in 
scene #Plan // null 



"John" / plans in 
scene #Plan // I / 
pick-up / the milk 



"John" / plans in 

scene #Plan // 1 / 

drink / the milk 



Scene: [] #Pli 



Scene #Plan / create 
-instance / milk. 




Milk / create 

-relation Inside / 

fridge. 



I / pick-up / the 
milk 



I / drink / the 
milk 



"John" / open / 
fi-idge. 



Fig. 7. 
place. 



The focus Vis after the creation of a plan. The plan is represented in a sepai'ate scene which describes the environment in which the plan is to take 



1 "John" / goes-to / a kitchen. 

2 The kitchen / is-identical / 

3 the kitchen — in -- "plan" . 

4 "John" / open / a fridge. 

5 The fridge / is-identical / 

6 the fridge — in -- "plan" . 

7 The fridge / is-a / empty. 

The relation between the plan and its execution is reflected in 
the identity relations: for instance, the kitchen into which John 
went is identity-related to the one in the plan. In the story snipped 
above, the identity relations are explicitly created in the story. In 
other circumstances, these relations can be inferred by the agent 
through a missing relation inference. It is also possible that a fast and 
careless reader (such as a Xapagy agent presented with a sequence 
of statements without interstitial pauses for the creation of headless 
shadows) would not even notice that what John is doing is actually 
an enactment of the plan. 

V. Unusual literary examples 

Let us now consider two more unusual examples of representation 
challenges, which illustrate the difficulty of translating complex 
expressions into Xapi. 

A. Cicero disrespects Caesar: same action dijferent meaning 

Greetings are formal exchanges between humans. While the greet- 
ing words normally have roots in comprehensible sentences, they 
are normally treated as opaque words. Let us consider the following 
exchange: 

Cicero and Caesar met each other. Caesar said "Ave!". 
Cicero replied "Salve!". 
The first approximation for the Xapagy agent is to simply observe 
what just happened. This can be done by recording the utterances in 
an unparsed natural language form. 

1 "Cicero" / meets / "Caesar". 

2 "Caesar" / utters / text "Ave!". 

3 "Cicero" / utters / text "Salve!". 

A next step would be to understand these two acts as greetings. 
For this, the agent does not need to perform a deep parse on the 
text: in fact, in the case of greetings, which are highly ritualized, 
conventional words, such deep parses are useless. The agent can 
recognize "Ave" and "Salve" as greeting words by simple matching. 
The interpretation of the act will be attached to the observed act 
through the coincidence relation. The interpretation, in this case, can 
be generate by the missing action mechanism. 

Note that the coincidence relation does not necessarily involve 
that the Vis have been created simultaneously, only that the Vis 
refer to different aspects of the same event. In this case, we have 
the observation and the interpretation of the event in the same 
coincidence group. As we shall see later, other combinations are also 
possible. The only restriction is that the VI must be still in the focus 
in order for the system to be able to add new Vis to its coincidence 
group. 

The resulting code can be written in Xapi as follows: 

1 "Cicero" / meets / "Caesar". 

2 "Cicero" / utters / text "Ave!". 

3 "Cicero" / thus greets / "Caesar" . 

4 "Caesar" / utters / text "Salve!". 

5 "Caesar" / thus greets / "Cicero" . 

In addition to the interpretation of the individual utterances as 
greetings, we can also summarize the whole exchange as a single 
summary action: 

1 "Caesar" + "Cicero" / 

2 in-summary meet-and-greet-each-other . 

Thus, the interpretation leads to the structure in Figure [9] 



B. Lawrence of Arabia: evolving identity through recognition 

We have seen that changes in the attributes in real world objects are 
be represented by the change of instances. There are also situations 
when the real world entity does not change, but the agent's changes 
its opinion about it (for instance, by correcting mistakes in attribute 
recognition). To illustrate such a case, let us consider the famous 
scene in Lawrence of Arabia, which is described on the website 
www.filmsite.org as follows: 

A dust cloud and then a tiny speck appear through shim- 
mering, mirage-like heat waves on the desert horizon - 
Lawrence fears it is "Turks." The ominous image, more 
mirage than real, steadily enlarges and grows into a human 
being as it comes closer and closer. 
Let us now model this situation in Xapi. To avoid mul- 
tiple quote sentences, we will write this from the point of 
view of Lawrence (to model it from the perspective of a third 
party observer, we can prefix each of these statements with 
"Lawrence" / believes in scene). 

1 A speck dust / is-far-away / "Me" . 

2 The speck dust / is-approaching / "Me". 

3 The speck dust / changes / mirage . 

4 The mirage / is-approaching / "Me". 

5 The mirage / changes / a turk man + a turk man. 

6 The turk man + the turk man / changes / a man. 

7 The man / is-approaching / "Me". 

8 The man / is-a / beduin. 

9 The man / is-a / black-dressed. 

In this case we have five changes of the instance which represent 
the entity to which Lawrence is looking. Note that the entity itself 
remains the same - there is a continuity of identity between the 
speck of dust first seen and the black-robed Beduin (Omar Sharif) 
who approaches and shots Lawrence's companion. This continuity 
is reflected through the chain of identities connecting the instances 
which represent the various interpretations: speck of dust, mirage, 
two Turks, one man. In fact, the number of instances created are 
even larger, because one of the interpretations actually refers to a 
group (two Turks). On the other hand, this example also illustrates 
that when a new discovery can be added as an additional attribute to 
the instance, we do not need to change the instance. 

Now, of course, the Lawrence of Arabia scene is a famous example 
of successive revisions of incompatible attributes. In real life, it is rare 
that the recognition process would proceed this slowly and through 
this many iterations. However, the gradual recognition of attributes 
(with or without the necessity of change) is an everyday occurrence. 
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Fig. 8. Mismatch between tlie plan and reality: there is no milk in the fridge. 
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Fig. 9. The focus Vis for the example of Caesar and Cicero greeting each other. 











Scene: ["Arabia"] 

Far 






Near_^ 








speck dust 










"Lawrence" 






► 


mirage 








" " - 


. - -r 


man #B #B 








' ■ - - . 
















- -k 


{man #T2 #T2 + man 
#T1 #T1} 






man #T2 #T2 


MemberOfGroup ^ 
















^^^-'^'^ 




man #T1 #T1 

























Fig. 10. The instances in the current scene for he case of the beduin being recognized by Lawrence. The shaded instances are in focus, the unshaded ones 
are not. 



